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Even simple behaviour requires us to make decisions based on combining multiple pieces of learned and new information. 
Making such decisions requires both learning the optimal response to each given stimulus as well as combining probabilistic 
information from multiple stimuli before selecting a response. Computational theories of decision making predict that learning 
individual stimulus-response associations and rapid combination of information from multiple stimuli are dependent on differ- 
ent components of basal ganglia circuitry. In particular, learning and retention of memory, required for optimal response choice, 
are significantly reliant on dopamine, whereas integrating information probabilistically is critically dependent upon functioning 
of the glutamatergic subthalamic nucleus (computing the 'normalization term' in Bayes' theorem). Here, we test these theories 
by investigating 22 patients with Parkinson's disease either treated with deep brain stimulation to the subthalamic nucleus and 
dopaminergic therapy or managed with dopaminergic therapy alone. We use computerized tasks that probe three cognitive 
functions — information acquisition (learning), memory over a delay and information integration when multiple pieces of se- 
quentially presented information have to be combined. Patients performed the tasks ON or OFF deep brain stimulation and/or 
ON or OFF dopaminergic therapy. Consistent with the computational theories, we show that stopping dopaminergic therapy 
impairs memory for probabilistic information over a delay, whereas deep brain stimulation to the region of the subthalamic 
nucleus disrupts decision making when multiple pieces of acquired information must be combined. Furthermore, we found that 
when participants needed to update their decision on the basis of the last piece of information presented in the decision-making 
task, patients with deep brain stimulation of the subthalamic nucleus region did not slow down appropriately to revise their 
plan, a pattern of behaviour that mirrors the impulsivity described clinically in some patients with subthalamic nucleus deep 
brain stimulation. Thus, we demonstrate distinct mechanisms for two important facets of human decision making: first, a role 
for dopamine in memory consolidation, and second, the critical importance of the subthalamic nucleus in successful decision 
making when multiple pieces of information must be combined. 
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Introduction 

Optimal decision making usually requires several factors to be 
taken into account and appropriately combined or integrated. 
For example, when an animal is about to pounce upon its prey, 
many items of information including size, speed and distance of 
the prey, in addition to the hunter's own prowess, must be accur- 
ately combined to increase the chance of success. Recent compu- 
tational models have proposed that separate neural populations 
within the basal ganglia are involved in learning probabilistic 
associations and combining probabilistic information. In this article, 
we assess these predictions by testing patients with Parkinson's 
disease ON and OFF deep brain stimulation (DBS) to the region 
of the subthalamic nucleus and ON and OFF dopaminergic 
medication. 

Dopamine is a modulatory neurotransmitter, widely involved in 
learning from feedback, flexible motor planning and decision 
making (Egelman et al., 1998; Welberg, 2009). Midbrain dopa- 
minergic neurons facilitate adaptation of behaviour according to 
upcoming reward or task demands (Hollerman and Schultz, 1998; 
Cools et al., 2001, 2003; Fiorillo et al., 2003). Along with such 
trial-by-trial learning effects, neuromodulation by dopamine can 
take place over a longer time scale. Repeated dopamine release 
from both ventral tegmental-hippocampal and ventral tegmental- 
striatal projections can enhance or diminish neural connections 
over time, a process vital for establishing new longer term mem- 
ories (Calabresi et al., 1997; Bethus et al., 2010). Hence, one 
might expect that the level of dopamine released, from the ter- 
minal plexus of midbrain pathways, will predict ability to retain 
newly learned information. 

Depletion of dopamine from both nigro- striatal and ventral 
tegmental-mesolimbic pathways is found in Parkinson's disease 
(German et al., 1989), a condition typically treated with medica- 
tions that increase dopamine receptor stimulation. There is evi- 
dence that acquisition (learning) of probabilistic stimulus- 
response associations from feedback is abnormal in unmedicated 
patients with Parkinson's disease (Knowlton et al., 1996; Halbig 
et al., 2004), and they are particularly impaired in learning from 
positive feedback (Frank et al., 2004). Here we investigate the 
effect of dopaminergic medications on retention of information 
and predict that memory for learned information should be 
affected by medications. 

One important characteristic of a decision-making network is 
inhibition of the prepotent response to each individual stimulus, 
thus avoiding rapid impulsive behaviour that does not weigh up all 
options. The subthalamic nucleus, a region within the basal gang- 
lia, has such an inhibitory capability. Subthalamic nucleus output 
results in suppression of motor programmes and consequent re- 
sponse delay, which is particularly pertinent when there is conflict 
between possible action choices (Frank et al., 2007; Isoda and 
Hikosaka, 2008; Hershey et al., 2010; Cavanagh et al., 2011; 
Rodriguez-Oroz et al., 2011). 



Computational modelling of decision-making generates predic- 
tions about the precise role of the subthalamic nucleus when po- 
tential action choices compete. It has been proposed that the 
cortico-basal-ganglia circuit computes probabilities of different 
possible motor responses being correct according to Bayes' the- 
orem (Bogacz and Larsen, 2011). This model assumes that the 
cortex, together with striatum, computes which response is most 
likely to be correct (Fig. 1A). The model additionally assumes that 
feedback from the subthalamic nucleus (via output nuclei and 
thalamus) subsequently normalizes cortical activity, such that cor- 
tical neurons selective for a particular response have activity dir- 
ectly proportional to the logarithm of the probability of responses 
being correct (Fig. 1B). In particular, the model assumes that the 
subthalamic nucleus, together with external segment of globus 
pallidus, compute the normalization term in Bayes' theorem. This 
is a simple computation involving a non-linear summation of cor- 
tical inputs (provided to the subthalamic nucleus through the 
hyperdirect pathway), and it has been demonstrated that neurons 
in the subthalamic nucleus and globus pallidus have connectivity 
and response properties ideally suited to perform this computation 
(Bogacz and Gurney, 2007). The details of the model are 
described in the online Supplementary material. 

Because the model predicts that the subthalamic nucleus is 
involved in updating probabilities on the basis of combined 
pieces of information, we hypothesized that disruption of informa- 
tion processing in the subthalamic nucleus owing to DBS should 
impair patients' ability to combine probabilistic information. DBS 
reversibly modifies activity in the region of the subthalamic nu- 
cleus in patients with Parkinson's disease and improves motor 
symptoms including tremor, bradykinesia, rigidity and off-state 
freezing (Deuschl et al., 2006; Weaver et al., 2009; Adrian 
et al., 2010; Williams et al., 2010). In Parkinson's disease, sub- 
thalamic nucleus activity is abnormal, particularly in dorsolateral 
regions (Rodriguez-Oroz et al., 2001), and the aim of DBS is to 
electrically modify aberrant dorsolateral subthalamic nucleus firing, 
helping to normalize motor function (Plaha et al., 2006). Cognitive 
processes occur in more ventral subthalamic nucleus, relatively less 
affected in Parkinson's disease (Hershey et al., 2010). However, 
given the several millimetre electrical field surrounding the tip of 
the electrodes, ventral subthalamic nucleus can also be affected by 
dorsolateral^ sited DBS, putatively leading to interference with 
cognitive function (Hershey et al., 2010). In terms of cognitive 
processing, therefore, we can view subthalamic nucleus DBS as 
a reversible means by which to alter subthalamic nucleus firing, 
and we have used this in vivo human model to inform us with 
regard to the role of the subthalamic nucleus in probabilistic 
choice. 

We investigate the effect of both dopamine and subthalamic 
nucleus activity modulation on behaviour by testing patients 
with Parkinson's disease treated with dopamine replacement ther- 
apy and/or DBS to the region of the subthalamic nucleus. We use 
cognitive tasks probing probabilistic learning, memory retention 
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Figure 1 Computational model of probabilistic accumulation of information (Bogacz and Larsen, 2011), adapted to the reasoning task 
described. Each row of circles denotes neural populations within the brain area indicated by a label. 'Integrators' describe the cortical 
integrator neurons accumulating evidence for motor actions; 'Sensory' denotes visual cortical neurons encoding presented stimulus; 'STN' 
denotes subthalamic nucleus; 'GP' denotes external segment of globus pallidus; and 'Output' denotes output nuclei of the basal ganglia, 
i.e. internal segment of globus pallidus and substantia nigra pars reticulata. Black and light grey circles denote neural populations selective 
for pressing left and right buttons, respectively. Arrows denote excitatory connections, and lines ending with circles denote inhibitory 
connections. Labels denote the quantities represented by neurons while probabilities of responses are updated on the basis of presented 
stimulus. The details of this computation are described in the Supplementary material. Briefly, the model assumes that initially (A) the 
logarithms of prior probabilities of responses are represented by cortical integrators. They are combined in striatum with the log-likelihoods 
of presented stimuli given the responses. The logarithms of combined probabilities are projected via output nuclei and thalamus back to 
cortical integrators (top labels in B). On this basis, the subthalamic nucleus-globus pallidus circuit computes the logarithm of the nor- 
malization term in the Bayes theorem, which is subtracted in the output nuclei, giving the logarithms of posterior probabilities of responses. 



and information integration in patients with Parkinson's disease. 
This task is similar to the weather prediction task, variations of 
which have been used in other human and animal studies 
(Knowlton et al., 1996; Yang and Shadlen, 2007), but it is mod- 
ified so that we can differentiate deficits in learning and memory 
from information integration impairment (Fig. 2). In order to tease 
apart the effect of dopaminergic medication from that of subtha- 
lamic nucleus stimulation, we tested patients ON and OFF both 
dopaminergic therapy and subthalamic nucleus DBS. 

Materials and methods 

Participants 

Twenty-two patients with Parkinson's disease were tested (11 treated 
with bilateral DBS and dopaminergic medications and 11 treated with 
dopaminergic medications but without DBS, Table 1). We attempted 
testing in a further two patients treated with DBS who managed to 
partially complete only one session and therefore we excluded these 
data. Fifteen healthy age-matched control participants were also re- 
cruited (Table 1). Ethics approval was granted by the North Bristol 
NHS Trust Research Ethics Committee and all participants gave written 
consent in accordance with the Declaration of Helsinki. 

Experimental procedure 

For DBS patients, our computerized cognitive tasks were administered 
during routine postoperative inpatient assessments. This meant that 



DBS could be switched off for a longer duration than has been stand- 
ard in other studies investigating cognitive effects of DBS. DBS was 
switched OFF an average of 10 h before testing 'OFF DBS' and was 
switched back ON for a minimum of 2 h before testing 'ON DBS'. DBS 
patients were also tested ON and OFF medication. Medication was 
stopped on average 15 h before testing in the OFF medication state 
(practically defined OFF) and taken for a minimum of 18 h before 
testing in the ON medication state. For patients only receiving dopa- 
minergic medication, i.e. those without DBS, medications were 
stopped an average of 18 h before testing OFF medications or had 
been taken for a minimum of 18 h before ON medications testing. 
Thus, DBS patients were tested in four conditions: (i) ON medica- 
tion/ON DBS; (ii) ON medication/OFF DBS; (iii) OFF medication/ON 
DBS; and (iv) OFF medication/OFF DBS. Patients were tested in the 
order OFF/OFF, ON DBS/OFF medication, OFF DBS/ON medication, 
according to clinical protocol. The 'ON/ON' session was performed 
either first or last to demonstrate that there were no significant prac- 
tice effects (Supplementary material). Medication-only patients were 
tested ON and OFF medications, and the order of testing was coun- 
terbalanced across patients. Four of 44 testing sessions in the DBS 
group did not happen due to patient fatigue or the DBS not being 
turned off or medications failing to be omitted as planned (sessions 
missed: three OFF DBS/ON medication, one ON DBS/OFF medication, 
plus one memory phase only missing in the ON/ON condition). 
Patient demographics, medications and clinical information are listed 
in Table 1. 

Behavioural assessments 

At each testing session, a Unified Parkinson's Disease Rating 
Scale (UPDRS) motor scale (Part III) subscore was obtained 
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Figure 2 (Ai) Learning stage: participants receive feedback after pressing a button in response to a visual stimulus. (Aii) Sample 
probabilities of positive feedback for different stimuli. Stimuli in the left and right upper rows yield higher probability of positive feedback 
after left and right button press, respectively. Probabilities of positive feedback after optimal responses are shown on the right of the panel 
e.g. for the mouse shown in A, participant had 70% probability of positive feedback after pressing the right button, and 30% chance of 
receiving positive feedback after pressing the left button. (B) Integration stage: participants selected the best response after seeing three 
stimuli consecutively (separated by a 50 ms pause). (C) Memory stage: participants were asked to choose left or right after visual stimulus 
according to what they had previously learned. No feedback was provided in integration or memory stages. 



(Fahn and Elton, 1987) (Table 1). DBS patients also undertook the 
AAattis Dementia Rating Scale (Matteau et al., 2011), whereas medi- 
cation only patients and normal participants did the Mini-Mental State 
Examination (Folstein et al., 1975). Further assessments included a 
trait impulsivity measure (Barratt Impulsiveness Scale, Patton et al., 
1995) and the Lille Apathy Rating Scale (Sockeel et al., 2006) 
(Table 1). 

Deep brain stimulation surgical 
procedures 

Patients were all at least 6 months postoperative for deep brain elec- 
trode placement. DBS surgery was carried out under general anaes- 
thesia using T 2 -weighted MRI -directed visualization of the target. 



Our target in all cases was the posterior subthalamic area including 
the caudal zona incerta nucleus and the posterior aspect of the sub- 
thalamic nucleus. The surgical technique has been described previously 
(Patel et al., 2007) and involves the use of guide tubes and plastic 
stylettes to verify the position of the target, which is confirmed with 
an intraoperative MRI. This is followed by replacement of the stylettes 
with DBS electrodes (Model 3389, Medtronic Inc.) and allows for 
image-guided targeting accuracy of within 1 mm. 

Cognitive testing 

Testing took place in a quiet room in Frenchay Hospital, North Bristol 
Trust. Stimuli were presented in the vertical midline on a laptop com- 
puter. Left and right hand responses were recorded using a Cedrus 
RB-834 response pad. On each trial of the task, the participants were 



Basal ganglia roles in decision making 



Brain 2012: 135; 3721-3734 I 3725 



Table 1 Patient and healthy participant demographics 



Parkinson's DBS group 



Parkinson's 
medication group 



Healthy 
participants 



Number 

Mean age (standard error) 

Duration of disease (years) 

Time since stimulator insertion (months) 

Levodopa equivalence (mg) 

Mini-Mental State Examination 

DRS2 AEMSS 

LARS apathy score 

BIS impulsivity score 

UPDRS motor 



11 

56 (2.3) 
9.5 (0.8) 
26.5 

944 (133) 
10.3 

-21 (2.7) 
71 

ON/ON 

ON DBS/OFF MEDS 
OFF DBS/ON MEDS 
OFF/OFF 



11.7 (2) 
18.6 (1.8) 
17.25 (3.3) 
43.6 (3.0) 



11 

58.9 (2.0) 
7.27 (1.65) 

902 (148) 
27.0 (0.9) 

-22.4 (3) 
58 

ON MEDS 18.8 (5.2) 

OFF MEDS 28.0 (7.6) 



15 

57 (3.4) 



29.1 (0.3) 



-28.6 (1.3) 
57 



BIS = Barratt Impulsiveness Scale (performed ON medication or ON medication and on DBS); DRS2 AEMSS = age-adjusted score for the dementia-rating 
scale: only scores >6 were included in the study; LARS = Lilly Apathy Rating Scale. Standard errors are in brackets. 



asked to choose between two response options (left or right hand 
button) that were associated probabilistically with presented visual 
information. 

In the first testing session, participants were given a practice task to 
familiarize themselves with the button response box and learning of 
probabilistic information. In the practice task (40 trials), participants 
were shown one of two octopuses: one with an 80% probability of 
left being the correct answer; the other with an 80% probability of 
right being the correct answer. Otherwise, the experimental procedure 
for the practice task was as for the learning task (described below). 

The main experiment comprised three stages: (i) Learning task; 
(ii) Integration task; and (iii) Memory task to allow differentiation of 
each participant's capacity for learning and remembering as well as 
integrating information to reach a decision when ON or OFF stimula- 
tion and/or ON or OFF medication. 



Cognitive testing 



Learning task 

On each trial of the learning task, participants were shown a single 
visual stimulus (cartoon animal) for 1 s. There were four different sti- 
muli in each task presented 30 times each across three blocks (10 
times per block) making a total of 120 trials (three blocks of 40 
trials). The interstimulus interval was 2 s plus reaction time, and the 
order of stimuli was pseudorandomized such that the number of dif- 
ferent stimuli per block was controlled. After the stimulus, participants 
were shown a GO signal to prompt them to choose either a left or 
right hand button response (Fig. 2Ai). On the basis of probabilistic 
feedback after the response, participants could learn the optimal 
button to press for a given stimulus (Fig. 2Aii). Stimuli and feedback 
parameters were modified after pilot investigations such that the even- 
tual paradigm used was optimized to discourage declarative learning of 
stimulus-response associations. The aim was that participants had a 
vague habitual sense or instinct for the correct response, and verbal 
strategies for learning information were minimized. Thus, we sought to 
investigate the type of 'automatic' decision making in which the basal 
ganglia are implicated (Redgrave et al., 2010). 

Four groups of four animals (mice, beetles, mosquitoes and fish) 
were used, and both the animals used in each condition and the 



meaning of each individual animal were counterbalanced across par- 
ticipants (mice shown in Fig. 2, other stimulus sets shown in 
Supplementary Fig. 1). Previous reports have suggested that subtha- 
lamic nucleus DBS may interfere with processing of stimuli evoking 
negative emotions such as fear or anger (Lawrence et al., 2002; 
Dujardin et al., 2004). Consequently, our stimuli were designed to 
avoid conveying intensely negative emotions and, furthermore, the 
counterbalancing of stimulus sets and order was chosen to avoid 
any confound from facial emotional processing deficits. 

Integration task 

Using the learned response associations mentioned above, participants 
were then asked to decide which button to press after seeing three 
stimuli in succession (Fig. 2B). A GO signal was presented after 
the third stimulus, and participants were informed that they must 
wait for the GO signal before responding. This was to ensure they 
viewed all three animals before responding. If an anticipatory response 
was made, the early response was not recorded and the GO signal 
remained on the screen until the participant responded again i.e. only 
responses made after seeing all three animals were analysed. Two 
blocks of 64 stimuli were presented. No feedback was given. 

It was anticipated that participants would combine data in the 
integration task according to Bayes' theorem (with equal prior 
probabilities): 

P(S,\L)P(S 2 \L)P(S 3 \L) 



P(L\Si,S 2 ,S 3 )- 



tP(S,\L)P(S 2 \L)P(S 3 \L) 

I +(1 - P(S, |L))(1 - P(S 2 |L))(1 - P(S 3 \L)) 



(D 



where P(S,\L) are the probabilities of leftward movement associated 
with the stimuli in the learning task. The trial was considered as correct 
if the participant chose left response when P(L\S^,S 2 ,S 3 ) > 0.5 or if the 
participant chose right response when P(L\S^,S 2 ,S 3 ) < 0.5. 

Memory task 

After the integration stage, the stimuli were shown again individually 
(as in the learning task), and participants were asked to respond left or 
right according to the previously performed learning task. In this 
memory task, however, in contrast to the learning task, no feedback 
was provided. Each stimulus was shown a total of five times giving a 
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total of 20 trials in a randomized order. The memory task was com- 
menced ~20min after the onset of the learning task. 

Data analysis 

Accuracy 

Accuracy rates (%) were calculated for each of the learning, integra- 
tion and memory tasks for each condition (ON or OFF DBS and/or ON 
or OFF dopaminergic medication) in each participant. Overall effects 
of DBS or medication were investigated according to our a priori 
hypotheses. In addition, we compared the average performance of 
patients (collapsed across different DBS and medication conditions) 
with that of healthy participants by performing three one-way 
ANOVAs for learning, integration and memory, with average accuracy 
as dependent variable and subject group as factor. 

We hypothesized that dopamine replacement medication would 
affect learning and memory or retention of information. To investigate 
this, we looked at the effects of medication on accuracy in the learning 
and memory parts of the task. Note that learning data presented are 
based on performance in the final (third) block of the task to reflect 
the patients' eventual accuracy in the task rather than the trial and 
error of the first blocks (see Supplementary material for comparison of 
performance in last block with that overall). We wanted to separate 
memory effects at 20min from immediate learning effects and so data 
used for our memory ANOVA were (memory-learning) scores in each 
condition for each individual patient. We performed a repeated meas- 
ure mixed-design ANOVA in SPSS (PASW 18.) for learning (last block) 
and memory (memory-learning) accuracy scores separately, with 
medications state (ON versus OFF) as the within participant factor 
and group (DBS or medication only) as the between participants 
factor. 

To present the percentage effect of medication on performance, we 
calculated an overall effect of medication on memory for each of the 
22 patients [average percentage difference between (memory-learn- 
ing) scores in ON compared with OFF medication states, collapsed 
across DBS state for the DBS patients]. We then looked to see whether 
the overall performance in the memory tasks in the 'dopaminergic 
medication only' patients differed from that of normal healthy control 
subjects. We used independent samples f-tests comparing ON medi- 
cation versus normal participants' memory accuracy and OFF medica- 
tion versus normal participants' memory accuracy separately. Potential 
effects of DBS on learning and memory were investigated with 
repeated measures ANOVA on DBS patient data for learning and 
memory tasks separately with medication and DBS state as within 
participant factors. 

We hypothesized that DBS would negatively impact on accuracy in 
the integration task. Mean integration accuracy results for each DBS 
patient, in each condition (ON and OFF both DBS and medication), 
were analysed using a mixed-design repeated measures ANOVA 
where the within participant variable was DBS (ON versus OFF) and 
the between participant factor was medication state (ON or OFF). 
Further, paired sample £-tests were performed on the integration ac- 
curacy scores ON DBS/ON medication versus OFF DBS/ON medication 
and ON DBS/OFF medication versus OFF DBS/OFF medication. To 
present the percentage effect of DBS on performance, we calculated 
an overall effect of DBS on integration for each of the 11 patients 
(average percentage difference integration accuracy scores in ON com- 
pared with OFF DBS states, collapsed across medication state). 

We also investigated whether DBS impaired performance on both 
easy trials (with substantial evidence for one of the alternatives) and 
difficult trials (with weaker or conflicting evidence). We split trials in 



the integration task on the basis of overall probability of left 
response given three stimuli (Equation 1) and refer to trials in which 
this probability was between 25% and 75% as difficult, and to all 
other trials as easy. Data are presented graphically (Supplementary 
Fig. 2A). 

Reaction times 

We investigated the effects of DBS on reaction time. Specifically, we 
were interested to see whether the normal delay evoked by the need 
to dramatically update a decision was affected by subthalamic nucleus 
DBS. For each participant and condition, we computed the difference 
between median reaction time on trials when the last stimulus should 
have prompted a different response from the previous two (e.g. left, 
left, right) and on trials when all of the stimuli predicted the same 
direction (e.g. left, left, left). We then compared the delay when DBS 
was ON compared with OFF using a repeated measures ANOVA with 
DBS ON versus OFF and medications ON versus OFF as within par- 
ticipant factors. We performed a secondary analysis to ensure that the 
delay was due to updating of the decision process, rather than being 
due to processing of contrasting visual material (Supplementary 
material). 

Results 

Dopamine replacement therapy 
ameliorates loss of procedural memory 

How did dopamine replacement impact on the ability to acquire 
and maintain memory? There was a trend towards average per- 
formance in both the DBS and medication-only patients being 
worse than that of healthy age-matched participants in the last 
block of the learning task [F(2,624) = 2.5, P = 0.1] and no clear 
difference between patients and healthy participants in the 
memory task [F(2,780) = 2.9, P = 0.71] (Fig. 3A and B). 

In patients, no consistent effects of dopamine replacement were 
found in the learning stage [F(1 ,83.2) = 0.4, P = 0.4; Fig. 3A]. 
However, unmedicated patients with Parkinson's failed to retain 
learnt information as well as medicated patients in the memory 
stage [significant effects of medication on memory in mixed- 
design ANOVA incorporating all patients: F(1 ,2560) = 1 1 .7, 
P < 0.005; Fig. 3B]. The average benefit of medications on 
memory accuracy when assessed at 20min was 15% (standard 
error 4.3% across all 22 patients). 

Accuracy of medication-only patients with Parkinson's disease 
(n = 11) on treatment in the memory task was not significantly 
different from that of normal participants [68% (SEM 5.8%) con- 
trol subjects versus 61% (SEM 5.3%) medicated patients with 
Parkinson's disease, P = 0.46]. However, the accuracy of patients 
with Parkinson's disease when unmedicated was significantly less 
than that of normal participants [68% (SEM 5.8%) control sub- 
jects versus 45.5% (SEM 4.3%) unmedicated patients, independ- 
ent samples Mest, £ = 2.1, P < 0.05]. Therefore, dopamine 
replacement appears to correct defective memory retention in 
Parkinson's disease when assessed at 20min. 
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Figure 3 Mean accuracy in the last block (40 trials) of learning stage (A) and in memory stage (B). The bar graphs on the left show the 
results from patients tested ON and OFF medication (MEDS) and DBS, whereas the bar graphs on the right show results from patients' 
tests ON and OFF medication. The colours of the bars indicate testing condition. Error bars give the standard error of the mean (SEM) 
difference in accuracy between DBS ON and OFF for ON and OFF medication conditions separately for the DBS group and average 
difference between ON and OFF medication accuracy for the medication-only group. "Statistically significant comparison between 
performance ON and OFF medication in the memory phase of the task. # Although 11 patients performed in the ON medication/ON DBS 
condition, one memory phase was omitted and hence there are only 10 patients in this condition (Graph B, column 1). 
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Deep brain stimulation to subthalamic 
nucleus selectively impairs 
probabilistic choice 

Average performance across conditions in the integration task was 
worse in patients compared with healthy participants [F(2,1124), 
P<0.05]. Post hoc analysis with Bonferroni correction demon- 
strates that both medication-only and DBS patients differ from 
healthy participants (P < 0.05 for both comparisons). 

We predicted that DBS to the subthalamic nucleus would impair 
choice on the basis of probabilistic information from multiple 
stimuli. Overall accuracy of integration was impaired in patients 
ON DBS compared with patients OFF DBS. On average, accuracy 
in the integration task was 11.5% (SEM 4.3%) higher when pa- 
tients were tested OFF DBS compared with ON DBS. There was a 
significant main effect of DBS on integration [F(1 ,573) = 4.9, 
P<0.05], but no interaction between the effects of DBS and 
medication (interaction dbs x meds) [F(1 ,176.8) = 0.669, 
P = 0.426]. There was no effect of DBS on learning or memory 
[F(1 ,0.598), P = 0.451 and P(1,291), P = 0.506, respectively] or 
interaction between medication and DBS on learning and 
memory. DBS effects were similar regardless of whether patients 
were ON or OFF medication while performing the integration task 
and remained significant when patients' ON medication scores 
alone were considered (£ = 2.6, P < 0.05) (Fig. 4A). 
Supplementary Fig. 2A shows that DBS affected performance simi- 
larly for both easy and difficult trials. 

Next we asked whether the patients ON DBS fail to slow down 
following conflicting stimulus. Normal participants slow down 
when the last animal shown in the integration task predicts a 
change of plan, that is, when the third stimulus strongly conflicts 
with the first two [average median reaction time when the last 
stimulus predicts different direction than previous two = 558 ms 
(SEM 63 ms); average median reaction time when all stimuli pre- 
dict the same direction = 474 ms (SEM 56 ms)]. It is proposed that 
such behavioural delay results from subthalamic nucleus activity, 
which slows down responses on high conflict trials (Frank et al., 
2007). Patients tested OFF DBS performed as normal participants, 
with an appropriate delay when the decision had to be updated 
(Fig. 4B). In contrast, when subthalamic nucleus DBS was switched 
ON, delay was significantly reduced (repeated measures ANOVA 
comparing delay when DBS was ON compared with OFF: main 
effect of DBS [P(1,0.2) = 31.1, P < 0.05], no main effect of medi- 
cation [P(1,0.16) = 1.14, P = 0.327]), in keeping with the hypoth- 
esis that the need to slow down in the face of conflicting 
information requires normal functioning of the ventral subthalamic 
nucleus. 

Simulations of a computational model 

We simulated the model of the cortico-basal-ganglia circuit (Fig. 1) 
adapted to our task (details of the simulation methods are described 
in the Supplementary material). Because in our task participants were 
required to respond after a cue, we followed a common approach to 
simulating choices in tasks with cued response (e.g. Usher and 
McClelland, 2001; Bogacz et al., 2006), and to determine a choice 



in a simulated trial, we simply assumed that the response correspond- 
ing to the most active integrator at the time of the cue is chosen. 

The effect of medication could be simulated in two ways. The 
first method corresponds to the hypothesis that medication influ- 
ences acquisition of new memories, and would involve lowering 
the rate of learning of cortico-striatal synapses in simulated OFF 
medication conditions. However, this would result in difference in 
the accuracies in the learning phases between medication condi- 
tions in the simulated data, which is not present in the patient 
data. The second method corresponds to the hypothesis that the 
medication influences retention of memories, and involved intro- 
duction of decay over time of cortico-striatal synaptic weights in 
simulated OFF medication conditions. The performance of the 
simulated model is shown in Fig. 5 and is qualitatively consistent 
with our patient data, as the accuracies in the memory phase 
(Fig. 5B), but not learning (Fig. 5A) phase, significantly differ be- 
tween medication conditions. 

The effect of DBS was simulated in the model by setting the 
activity level in the subthalamic nucleus to 0. As we describe in 
detail below, this manipulation impairs accuracy in the integration 
task in the model, because the neural populations normally have 
activities limited to certain ranges, and lack of subthalamic nucleus 
modulations permits the activities to reach limits of these ranges, 
which owing to physical limitations distorts encoded information. 
Although subthalamic nucleus neurons are excitatory (glutama- 
tergic), the overall effect of their activity on cortex is inhibitory 
because the polysynaptic projection from the subthalamic nucleus 
to cortex involves inhibitory neurons (Fig. 1). Thus, a decrease of 
subthalamic nucleus activity in the simulated DBS ON condition 
results in an increase after each stimulus of activity of both cortical 
integrator neurons representing left response and the neurons rep- 
resenting right response. 

In the integration phase, in which multiple stimuli are presented, 
the activity of integrators may exceed the value representing prob- 
ability of 1, i.e. exceed their normal range of activity (as a prob- 
ability is by definitional). So high cortical activity may result in 
output nuclei getting negative activity in simulations, but because 
firing rate cannot be negative, we set it to 0 whenever high in- 
hibition from the integrators (via striatum) is received. This bound- 
ing of activity of output nuclei was done both in simulation of DBS 
ON and DBS OFF state (although it had little effect in simulated 
DBS OFF condition). We note that in the real brain, when output 
nuclei get activity close to 0, usually action is initiated, but because 
the task requires refraining from the response until the GO signal, 
there may be additional processes preventing a premature re- 
sponse (and possibly increasing activity in output nuclei) that we 
do not include in our model. This bounding mostly limits the ac- 
tivity of the integrator corresponding to the correct response (as it 
receives more input), and hence reduces the difference between 
the two integrators and thus the probability of making the correct 
choice. Consequently, the model makes more errors in the inte- 
gration phase in simulated DBS ON conditions (Fig. 5B). 
Supplementary Fig. 2B shows that simulated effect of DBS im- 
paired performance on both easy and difficult trials, which were 
defined in Supplementary Fig. 2A illustrating a similar effect in 
patients. This suggests that the boundary described above may 
be reached in simulations of both easy and difficult trials. 
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Figure 4 Effects of DBS and medication on performance in the integration task. The colours of the bars indicate the testing condition. 
(A) Mean accuracy. Error bars show the SEM. (B) The average median reaction time in the trials on which the last stimulus predicts the 
opposite response to the first and second (incompatible), and the trials on which all three stimuli predict the same response (compatible). 
For each combination of DBS and medication state, the error bars on both columns show the standard error of the differences between 
reaction times on trials where the last stimulus is incompatible compared with compatible. ^Significant statistical comparison: patients are 
more accurate in the integration task when DBS is switched OFF compared with DBS ON. 
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Figure 5 Performance of computational model. The colours of the bars indicate the simulation condition. Error bars show the SEM. The 
mean accuracy of the model in learning (A), memory (B) and integration (C) phases. "Statistical significance analogous to behavioural 
results: patients ON DBS are impaired in the integration task (C) and dopaminergic medication improves memory (B). 



Nevertheless, in the simulated DBS on condition, the accuracy is 
higher for easier trials, because the integrator corresponding to the 
incorrect response gets less input in the easy than in the difficult 
condition, and hence is more likely to have activity further below 
that of the integrator corresponding to the correct response. 



stimulated, participants did not slow down sufficiently to allow 
the appropriate updating of information mandated for an optimal 
decision to be reached. Thus, we infer that the subthalamic nu- 
cleus is a critical component of the pathway that serves the func- 
tion of accumulating and integrating probabilistic information. 



Discussion 

We sought to investigate the role of dopamine and the subthala- 
mic nucleus within the network of brain areas involved in prob- 
abilistic decision making. We found that dopamine replacement 
therapy influenced memory for learned response associations. 
Dopamine replacement boosted performance in our memory 
task, invoking a role for dopamine in procedural memory reten- 
tion. In contrast, switching ON subthalamic nucleus DBS resulted 
in a specific probabilistic choice deficit and no significant effect on 
memory. Thus we demonstrate the pivotal role of the subthalamic 
nucleus in permitting the successful integration of information 
required for probabilistic decisions. 

To perform the integration stage of the task, participants had to 
accumulate and weigh information from three sequentially, but 
rapidly, presented stimuli. Optimal performance required updating 
of the decision process as each stimulus was presented. Reaction 
time analysis showed that when the subthalamic nucleus was 



Effect of dopaminergic medications 

How does dopamine therapy improve retention of longer term 
memories? First, it is important to note that converging evidence 
points to multiple parallel memory subsystems within the human 
brain (Poldrack and Packard, 2003). The basal ganglia are centrally 
involved in encoding procedural memory or stimulus-driven action, 
rather than more explicit or declarative information (Knowlton 
et al., 1996). Here, we are making a further distinction between 
very short-term memory formation over a few seconds or very 
few minutes and medium or longer term memory retention over 
several minutes or hours (Smith et al., 2006). Our data support a 
role for dopamine in medium- to long-term retention of proced- 
ural or stimulus-driven memory. One could consider whether it is 
truly memory retention and not recall that is affected by dopa- 
mine, particularly as one previous study has suggested that dopa- 
mine is required for recall of information in a novel context 
(Shiner et al., 2012). However, no such novelty applies in the 
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memory phase of our task as stimuli and task requirements are as 
for the learning phase. 

Dopamine modulates neuronal firing patterns after repeated 
stimulation, resulting in long-term potentiation and depression 
(Calabresi et al., 1997; Bethus et al., 2010), processes thought 
to be important for longer term memory retention (Cetonze 
et al., 2001; Lisman and Grace, 2005; Shohamy and Adcock, 
2010). Dopaminergic networks within basal ganglia and 
hippocampal-ventrotegmental loops have been implicated in 
laying down new longer term memories and habit formation 
(Redgrave et al., 2010). Given that dopamine depletion occurs 
in both these regions in Parkinson's disease (German et al., 
1989), one might expect that difficulty laying down longer term 
procedural memories contributes to the failure of automatic be- 
haviour commonly found in patients with Parkinson's disease 
(Yin and Knowlton, 2006). Replacement of dopamine within 
depleted midbrain networks may well be responsible for improve- 
ment in longer term memory in our task. 

Another possibility for why the withdrawal of medications 
affected accuracy in memory task but not in the learning task 
may be formulated on the basis of results from Frank et al. 
(2004). They showed that withdrawing dopaminergic medication 
disrupts specifically the ability to learn from positive feedback, and 
that unmedicated patients have an enhanced ability to learn from 
negative feedback. It may be possible that the lack of feedback in 
the integration phase, together with lowered level of dopamine, 
may drive synaptic plasticity which is normally occurring after 
negative feedback resulting in extinction of associations acquired 
in the learning phase. This hypothesis predicts that the effect of 
medications should disappear if the integration phase is replaced 
by a delay period. 

The extinction hypothesis described above could explain dis- 
crepancy between the effects of medications on accuracy we 
observed and the lack of overall effect observed by Frank et al. 
(2004) because their task did not include as prolonged a phase 
without feedback as in our task. However, there is another 
explanation for the above discrepancy in overall effect of medica- 
tions. The analysis of learning from positive and negative feedback 
in our task (Supplementary material) suggests that in all conditions 
participants predominantly learned only from positive, rather than 
negative feedback. It is possible that the lack of learning from 
negative feedback in our task is caused by a particular choice of 
wording 'Bad Luck' (Fig. 2A), which may suggest that the nega- 
tive outcome is not resulting from the participant's action. Because 
our participants predominantly learnt from positive feedback, the 
impaired performance in the memory phase OFF medications is 
consistent with the result of Frank et al. (2004), the critical differ- 
ence being that the memory effects of medication in our task were 
delayed rather than occurring at the initial learning stage. 

Effect of deep brain stimulation 

Topographically organized signals travel through the subthalamic 
nucleus, which is extensively interconnected via a wide arboriza- 
tion with other areas of the basal ganglia, including the globus 
pallidus, substantia nigra and striatum as well as cerebral cortex 
(Parent and Hazrati, 1995). A recent model of the basal ganglia 



proposes that the subthalamic nucleus is critically involved in the 
accumulation of probabilistic information (Bogacz and Gurney, 
2007). Our behavioural data are in accordance with the predic- 
tions of the model; here we demonstrate that altering subthalamic 
nucleus activity through DBS interferes with upcoming response 
choice when several factors need to be taken into account. 

Most of us take time when we change our minds, or update a 
decision according to new events. We have shown that the sub- 
thalamic nucleus is essential to allow us to slow down and revise 
response choice. Impaired reasoning following alteration of the 
subthalamic nucleus firing pattern by DBS stems from a failure 
to accrue information over time and is associated with an inappro- 
priate speeding of the response when new information should 
conflict with an already prepared motor programme. Thus, we 
have provided evidence that could explain how millisecond alter- 
ations in reaction times, shown in patients with subthalamic nu- 
cleus DBS (Frank et al., 2007), may underlie abnormal reasoning 
over a longer time frame (Frank et al., 2007; Wylie et al., 2010; 
Rodriguez-Oroz et al., 2011). Such breakdown in the ability to 
slow and integrate evidence before making a decision also helps to 
explain why some patients with subthalamic nucleus DBS clinically 
manifest abnormal behaviour including impulsivity (Voon et al., 
2008; Halbig et al., 2009). 

One may consider whether anything other than information 
integration deficit could account for the observed failure in our 
integration task. There was no consistent effect of subthalamic 
nucleus DBS on learning or memory of stimulus-response associ- 
ations. Therefore, neither learning nor memory deficits can explain 
the effects on integration. Impaired visuomotor processing speed 
has been suggested in patients with subthalamic nucleus DBS 
(Follett et al., 2010). We therefore performed a secondary analysis 
to see whether the normal delay associated with a dramatic 
change of decision at the final stimulus was actually a result of 
slowed visual, rather than motor, processing (Supplementary ma- 
terial). When the last two stimuli both suggested the same re- 
sponse, but were visually different however, there was no 
greater delay. Hence, the delay truly appears to represent slowing 
in the face of a more complex decision, rather than slowing pro- 
voked by more complex visual processing. Furthermore, if reduced 
visuomotor processing speed influenced our results, one would 
expect reaction times to be slower when participants performed 
more poorly, that is when DBS was switched ON. There were no 
significant effects of DBS on median overall reaction time 
(Supplementary material), but a trend towards faster median over- 
all reaction times when DBS was ON, making visuomotor process- 
ing speed very unlikely to be a factor in the poorer probabilistic 
choice observed. 

Can overall motor performance explain our results? Clearly both 
subthalamic nucleus DBS and dopaminergic medications improve 
motor symptoms in patients with Parkinson's disease (Deuschl 
et al., 2006; Weaver et al., 2009). Indeed, as expected, UPDRS 
scores showed improvement when ON both medications and DBS, 
with intermediate scores when one or the other was ON alone 
and with highest scores (most impaired) when OFF both medica- 
tion and DBS. In the integration task, however, poor performance 
actually occurred when subthalamic nucleus DBS was switched 
ON and the patients had relatively improved UPDRS scores. In 
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addition, our observed differential effects of medication and DBS 
on our integration and memory tasks, despite the fact that both 
manipulations alleviate motor dysfunction, are also against our 
behavioural results being explained on the basis of variations in 
motor function. In addition, neither could the specific probabilistic 
choice deficit be explained by generic failure of executive function 
such as perseveration, as a perseverative tendency would affect 
performance in both the learning and memory phases of our task 
as well as the integration phase. 

Our results may seem inconsistent with those of Halbig et al. 
(2004) who found better performance with DBS ON than OFF in 
the original weather prediction task in which learning and integra- 
tion are not separated into different stages of the experiment. 
Because most trials in their experiment involved both learning 
from feedback and combining of probabilistic information, it is 
not clear which of these cognitive tasks was affected by DBS. 

Relationship to the conflict model 

A computational model proposed by Frank (2006) has been very 
successful in explaining the effects of DBS on behaviour in choice 
tasks in which participants were free to respond at any time after 
stimulus onset. The model suggests that the subthalamic nucleus 
computes conflict between evidence representing alternative 
choices, and its influence on the output nuclei delays choice on 
more conflicting trials until the conflict resolves. Thus, the model 
naturally explains how the DBS reduces difference between reac- 
tion times on high and low conflict trials (Fig. 4B) (Frank et al., 
2007). More recently, it has been shown that increased activity of 
the subthalamic nucleus has a similar effect to an increase in de- 
cision threshold in a very influential abstract model of decision 
making — the diffusion model (Cavanagh et al., 2011; Ratcliff 
and Frank, 2012). The threshold parameter describes how much 
evidence needs to be gathered before choice, and lowering the 
threshold reduces both accuracy and reaction time. Within this 
framework, the conflict model would also explain the effect of 
DBS on accuracy in the integration task (Fig. 4A) by postulating 
that the reduced subthalamic nucleus activity corresponds to 
reduced threshold and hence lower accuracy. However, we do 
not see how the conflict model could account for the effect of 
DBS on accuracy being only significant in the integration task but 
not other tasks in our study. By contrast, the model described in 
this paper was not extended to capture the effect of DBS on 
reaction time in our task (it would be an interesting direction for 
future work), but it does explain the selective effect of DBS on 
accuracy in the integration task. In particular, it suggests that in 
the integration task, in which more stimuli were presented than in 
other tasks, the boundaries on the range of allowed activity are 
more likely to be reached due to longer lack of subthalamic 
nucleus normalization. Reaching these boundaries reduces the dif- 
ference between activities of neurons selective for the two 
responses and thus reduces the probability of the correct response. 

For most currently used choice tasks, the conflict model (Frank, 
2006) and the Bayesian model (Bogacz and Gurney, 2007; Bogacz 
and Larsen, 2011), that we adapted in this paper for our task, 
make similar predictions on the activity of the subthalamic nucleus, 
because the normalization provided by the subthalamic nucleus 



according to the Bayesian model is higher on high conflict trials 
(due to the mathematical definition of the normalization - see 
definition of SG(t) in Figure 3b in Bogacz & Larsen, 2011). 
Consequently, Zaghloul et al. (2012), who observed higher activ- 
ity of the subthalamic nucleus on neurons on high conflict trials, 
wrote that their results were predicted by both models. We feel 
that to investigate differences between these models, one will 
need to design new tasks for which the models make very differ- 
ent predictions. 

Conclusions 

We have demonstrated that response choice requiring information 
integration is impaired as a result of subthalamic nucleus DBS and 
associated with a failure to slow down in order to incorporate new 
information before taking a decision. Therefore, our data are con- 
sistent with the model of probabilistic decision making where the 
subthalamic nucleus is critical for the Bayesian normalization of 
accumulated information. Furthermore, we have uncovered an 
effect of dopamine depletion on memory retention > 20 min, 
which is somewhat ameliorated by dopamine replacement. In 
addition to illuminating the neurobiology and the role of the sub- 
thalamic nucleus in decision-making networks, these data may 
have clinical implications including the potential for use of dopa- 
mine as a therapeutic agent in conditions where memory retention 
is impaired. 
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